Skip to main content

Weaviate

Summary​

This document covers the information to gather from Weaviate in order to configure a Qarbine data service. The data service will use the Qarbine Weaviate driver. You can define multiple data services that access the same Weaviate endpoint though with varying credentials. Once a data service is defined, you can manage which Qarbine principals have access to it and its associated Weaviate data. A Qarbine administrator has visibility to all data services.

Overview​

Weaviate is a vector database for AI applications available as a cloud service. More information can be found at https://weaviate.io/

Weaviate Configuration​

For Qarbine to access your Weaviate data it needs:

  • a Weaviate endpoint,
  • a Weaviate API key and
  • a generative API key such as from OpenAI.

Open a web browser to https://console.weaviate.cloud/.

Sign on to your account.

The Weaviate Dashboard page is shown which includes a list of your clusters.

  

Locate your desired cluster and to its right click on the red highlighted icon shown below.

  

Copy the Rest endpoint URL.

  

Paste it in a temporary spot for use very soon. The value will be used in the Qarbine as the “server template” setting.

Click

  

In the dialog click the red highlighted icon,

  

Paste it in a temporary spot for use very soon. The value will be used in the Qarbine server options as

apiKey=’yourWeaviateApiKey’

Close the dialog by clicking

  

Qarbine Configuration​

Compute Node Preparation​

Determine which compute node service endpoint you want to run this data access from. That URL will go into the Data Service’s Compute URL field. Its form is “https://domain:port/dispatch”. A sample is shown below.

  

The port number corresponds to a named service endpoint configured on the given target host. For example, the primary compute node usually is set to have a ‘main’ service. That service’s configuration is defined in the ˜./qarbine.service/config/service.main.json file. Inside that file the following driver entry is required

"drivers" :[
 . . .
"./driver/weaviateDriver.js"
]

If you end up adding that entry then restart the service via the general command line syntax

pm2 restart <service>

For example,

pm2 restart main

or simply

pm2 restart all

Data Service Definition​

Open the Administration Tool.

Navigate to the Data Services tab.

  

A data service defines on what compute node a query will run by default along with the means to reach to target data. The latter includes which native driver to use along with settings corresponding to that driver. Multiple Data Sources can reference a single Data Service. The details of any one Data Service are thus maintained in one spot and not spread out all over the place in each Data Source. The latter is a maintenance and support nightmare.

To begin adding a data service click

  

On the right hand side enter a name and optionally a description.

  

Set the Compute URL field based on the identified compute node above. Its form is “https://domain:port/dispatch”. A sample is shown below.

  

Also choose the “Weaviate” driver.

  

The server template is the Weaviate cluster URL you copied above. An example is shown below.

  

You can reference environment variables using the syntax %NAME%. Any strings should be quoted and the key\value pairs separated by commas.

The server options include your Weaviate API key copied from above and your generative API key such as from OpenA.The general form is

apiKey = ‘yourWeaviateApiKey’,
headers={ 'X-OpenAI-Api-Key': 'yourOpenAiApiKey' }

These values are quoted and separated by a comma as shown below.

  

Test your settings by clicking on the toolbar image highlighted below.

  

The result should be

  

Save the Data Service by clicking on the image highlighted below.

  

The data service will be known at the next log on time. Next, see the Weaviate query interaction and any tutorial for information on interacting with Weaviate from Qarbine.

References​

See https://console.weaviate.cloud/documentation for more information.